from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.run()

KNeighborsClassifier_brute_force

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.102	0.163	0.0	0.019	-1	1	0.663	2.377	0.140	8.036	0.478	See	See
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.102	0.163	0.0	0.019	-1	1	0.663	0.029	0.005	651.321	111.600	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.349	0.009	0.0	0.349	-1	1	1.000	2.377	0.140	0.147	0.009	See	See
3	KNeighborsClassifier_brute_force	predict	100000	1	100	0.349	0.009	0.0	0.349	-1	1	1.000	0.029	0.005	11.902	2.057	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.082	0.330	0.0	0.019	-1	5	0.757	3.241	0.112	5.887	0.227	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.082	0.330	0.0	0.019	-1	5	0.757	0.028	0.003	679.022	64.791	See	See
6	KNeighborsClassifier_brute_force	predict	100000	1	100	0.357	0.013	0.0	0.357	-1	5	1.000	3.241	0.112	0.110	0.005	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1	100	0.357	0.013	0.0	0.357	-1	5	1.000	0.028	0.003	12.702	1.274	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.609	0.133	0.0	0.020	1	100	0.882	2.259	0.027	8.681	0.120	See	See
9	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.609	0.133	0.0	0.020	1	100	0.882	0.022	0.001	877.951	23.499	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1	100	0.353	0.014	0.0	0.353	1	100	1.000	2.259	0.027	0.156	0.007	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.353	0.014	0.0	0.353	1	100	1.000	0.022	0.001	15.804	0.752	See	See
12	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.001	0.017	0.0	0.019	-1	100	0.882	3.267	0.064	5.816	0.115	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.001	0.017	0.0	0.019	-1	100	0.882	0.027	0.002	696.061	53.757	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.346	0.007	0.0	0.346	-1	100	1.000	3.267	0.064	0.106	0.003	See	See
15	KNeighborsClassifier_brute_force	predict	100000	1	100	0.346	0.007	0.0	0.346	-1	100	1.000	0.027	0.002	12.685	1.014	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.101	0.002	0.0	0.019	1	5	0.757	2.247	0.030	8.500	0.112	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.101	0.002	0.0	0.019	1	5	0.757	0.022	0.002	867.183	67.801	See	See
18	KNeighborsClassifier_brute_force	predict	100000	1	100	0.344	0.010	0.0	0.344	1	5	1.000	2.247	0.030	0.153	0.005	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1	100	0.344	0.010	0.0	0.344	1	5	1.000	0.022	0.002	15.635	1.307	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.964	0.045	0.0	0.019	1	1	0.663	1.355	0.011	14.000	0.123	See	See
21	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.964	0.045	0.0	0.019	1	1	0.663	0.024	0.004	804.866	142.226	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1	100	0.356	0.014	0.0	0.356	1	1	1.000	1.355	0.011	0.263	0.011	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	100	0.356	0.014	0.0	0.356	1	1	1.000	0.024	0.004	15.096	2.732	See	See
24	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.418	0.034	0.0	0.004	-1	1	0.896	2.111	0.080	2.093	0.081	See	See
25	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.418	0.034	0.0	0.004	-1	1	0.896	0.010	0.004	444.756	160.140	See	See
26	KNeighborsClassifier_brute_force	predict	100000	1	2	0.296	0.011	0.0	0.296	-1	1	1.000	2.111	0.080	0.140	0.007	See	See
27	KNeighborsClassifier_brute_force	predict	100000	1	2	0.296	0.011	0.0	0.296	-1	1	1.000	0.010	0.004	29.820	10.789	See	See
28	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.458	0.041	0.0	0.004	-1	5	0.922	2.916	0.080	1.529	0.044	See	See
29	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.458	0.041	0.0	0.004	-1	5	0.922	0.011	0.003	416.628	132.769	See	See
30	KNeighborsClassifier_brute_force	predict	100000	1	2	0.291	0.013	0.0	0.291	-1	5	1.000	2.916	0.080	0.100	0.005	See	See
31	KNeighborsClassifier_brute_force	predict	100000	1	2	0.291	0.013	0.0	0.291	-1	5	1.000	0.011	0.003	27.162	8.734	See	See
32	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.462	0.036	0.0	0.004	1	100	0.929	2.030	0.033	2.198	0.040	See	See
33	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.462	0.036	0.0	0.004	1	100	0.929	0.003	0.000	1395.329	114.673	See	See
34	KNeighborsClassifier_brute_force	predict	100000	1	2	0.292	0.012	0.0	0.292	1	100	1.000	2.030	0.033	0.144	0.006	See	See
35	KNeighborsClassifier_brute_force	predict	100000	1	2	0.292	0.012	0.0	0.292	1	100	1.000	0.003	0.000	91.210	8.308	See	See
36	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.529	0.058	0.0	0.005	-1	100	0.929	2.971	0.098	1.524	0.054	See	See
37	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.529	0.058	0.0	0.005	-1	100	0.929	0.012	0.004	387.254	132.223	See	See
38	KNeighborsClassifier_brute_force	predict	100000	1	2	0.291	0.007	0.0	0.291	-1	100	1.000	2.971	0.098	0.098	0.004	See	See
39	KNeighborsClassifier_brute_force	predict	100000	1	2	0.291	0.007	0.0	0.291	-1	100	1.000	0.012	0.004	24.858	8.502	See	See
40	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.414	0.045	0.0	0.004	1	5	0.922	2.047	0.029	2.156	0.037	See	See
41	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.414	0.045	0.0	0.004	1	5	0.922	0.003	0.000	1431.813	86.041	See	See
42	KNeighborsClassifier_brute_force	predict	100000	1	2	0.291	0.007	0.0	0.291	1	5	1.000	2.047	0.029	0.142	0.004	See	See
43	KNeighborsClassifier_brute_force	predict	100000	1	2	0.291	0.007	0.0	0.291	1	5	1.000	0.003	0.000	94.522	6.029	See	See
44	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.440	0.032	0.0	0.004	1	1	0.896	1.142	0.016	3.889	0.062	See	See
45	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.440	0.032	0.0	0.004	1	1	0.896	0.002	0.000	1932.338	382.374	See	See
46	KNeighborsClassifier_brute_force	predict	100000	1	2	0.291	0.010	0.0	0.291	1	1	1.000	1.142	0.016	0.255	0.010	See	See
47	KNeighborsClassifier_brute_force	predict	100000	1	2	0.291	0.010	0.0	0.291	1	1	1.000	0.002	0.000	126.539	25.403	See	See

KNeighborsClassifier_kd_tree

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	145.746	0.000	0.0	0.146	-1	1	0.929	0.547	0.016	266.246	7.857	See	See
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	145.746	0.000	0.0	0.146	-1	1	0.929	0.004	0.002	36778.445	18011.103	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.204	0.078	0.0	3.204	-1	1	1.000	0.547	0.016	5.852	0.224	See	See
3	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.204	0.078	0.0	3.204	-1	1	1.000	0.004	0.002	808.420	396.388	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	144.760	0.000	0.0	0.145	-1	5	0.946	1.029	0.021	140.694	2.875	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	144.760	0.000	0.0	0.145	-1	5	0.946	0.004	0.000	34437.147	3895.732	See	See
6	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.143	0.119	0.0	3.143	-1	5	1.000	1.029	0.021	3.055	0.132	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.143	0.119	0.0	3.143	-1	5	1.000	0.004	0.000	747.679	89.221	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	162.295	0.000	0.0	0.162	1	100	0.951	6.248	0.051	25.974	0.214	See	See
9	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	162.295	0.000	0.0	0.162	1	100	0.951	0.004	0.001	42524.017	8223.555	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.161	0.086	0.0	3.161	1	100	1.000	6.248	0.051	0.506	0.014	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.161	0.086	0.0	3.161	1	100	1.000	0.004	0.001	828.340	161.759	See	See
12	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	170.348	0.000	0.0	0.170	-1	100	0.951	3.288	0.031	51.811	0.494	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	170.348	0.000	0.0	0.170	-1	100	0.951	0.008	0.005	21062.762	13912.185	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.188	0.104	0.0	3.188	-1	100	1.000	3.288	0.031	0.970	0.033	See	See
15	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.188	0.104	0.0	3.188	-1	100	1.000	0.008	0.005	394.227	260.708	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	168.187	0.000	0.0	0.168	1	5	0.946	1.812	0.026	92.842	1.353	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	168.187	0.000	0.0	0.168	1	5	0.946	0.002	0.000	91756.111	18681.177	See	See
18	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.141	0.078	0.0	3.141	1	5	1.000	1.812	0.026	1.734	0.050	See	See
19	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.141	0.078	0.0	3.141	1	5	1.000	0.002	0.000	1713.381	351.396	See	See
20	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	170.030	0.000	0.0	0.170	1	1	0.929	0.966	0.021	175.964	3.743	See	See
21	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	170.030	0.000	0.0	0.170	1	1	0.929	0.001	0.000	138217.145	30676.423	See	See
22	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.224	0.175	0.0	3.224	1	1	1.000	0.966	0.021	3.337	0.195	See	See
23	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.224	0.175	0.0	3.224	1	1	1.000	0.001	0.000	2620.958	598.857	See	See
24	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.050	0.014	0.0	0.000	-1	1	0.891	0.032	0.005	1.571	0.492	See	See
25	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.050	0.014	0.0	0.000	-1	1	0.891	0.003	0.001	14.947	5.591	See	See
26	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	1	1.000	0.032	0.005	0.192	0.029	See	See
27	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	1	1.000	0.003	0.001	1.830	0.468	See	See
28	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.049	0.004	0.0	0.000	-1	5	0.911	0.035	0.009	1.402	0.372	See	See
29	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.049	0.004	0.0	0.000	-1	5	0.911	0.003	0.000	16.948	2.010	See	See
30	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	5	1.000	0.035	0.009	0.175	0.044	See	See
31	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	5	1.000	0.003	0.000	2.118	0.173	See	See
32	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.072	0.007	0.0	0.000	1	100	0.894	0.048	0.010	1.495	0.337	See	See
33	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.072	0.007	0.0	0.000	1	100	0.894	0.001	0.000	83.216	16.367	See	See
34	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	100	1.000	0.048	0.010	0.130	0.028	See	See
35	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	100	1.000	0.001	0.000	7.260	1.314	See	See
36	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.071	0.003	0.0	0.000	-1	100	0.894	0.047	0.005	1.531	0.173	See	See
37	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.071	0.003	0.0	0.000	-1	100	0.894	0.003	0.000	26.384	2.926	See	See
38	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	100	1.000	0.047	0.005	0.133	0.015	See	See
39	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	100	1.000	0.003	0.000	2.286	0.246	See	See
40	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.047	0.001	0.0	0.000	1	5	0.911	0.031	0.002	1.537	0.116	See	See
41	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.047	0.001	0.0	0.000	1	5	0.911	0.001	0.000	57.446	9.906	See	See
42	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.001	0.0	0.006	1	5	1.000	0.031	0.002	0.211	0.023	See	See
43	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.001	0.0	0.006	1	5	1.000	0.001	0.000	7.903	1.492	See	See
44	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.047	0.002	0.0	0.000	1	1	0.891	0.028	0.002	1.684	0.129	See	See
45	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.047	0.002	0.0	0.000	1	1	0.891	0.001	0.000	56.122	14.648	See	See
46	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	1	1.000	0.028	0.002	0.227	0.017	See	See
47	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	1	1.000	0.001	0.000	7.547	1.971	See	See

HistGradientBoostingClassifier_best

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	n_iter	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	predict	100000	1000	100	0.518	0.016	300	0.002	0.001	0.824	0.165	0.006	3.136	0.149	See	See
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	0.518	0.016	300	0.002	0.001	0.824	0.023	0.001	22.755	1.553	See	See
2	HistGradientBoostingClassifier_best	predict	100000	1	100	0.432	0.013	300	0.000	0.432	1.000	0.165	0.006	2.614	0.126	See	See
3	HistGradientBoostingClassifier_best	predict	100000	1	100	0.432	0.013	300	0.000	0.432	1.000	0.023	0.001	18.964	1.302	See	See